External-world recognition system

ABSTRACT

An external-world recognition system includes: a satellite positioning device that measures the position of a vehicle by receiving radio waves transmitted from artificial satellites; a yaw rate sensor that detects or estimates the movement of the vehicle; a camera that acquires external-world information about the surroundings of the vehicle; and an external-world recognition device that recognizes external-world information considering a reference direction as the center. The external-world recognition device corrects a detection value from the yaw rate sensor on the basis of time-series positions of the vehicle as measured by the satellite positioning device and corrects the reference direction on the basis of the corrected detection value.

TECHNICAL FIELD

The present invention relates to an external environment recognitionsystem (external-world recognition system) that acquires externalenvironment information around a vehicle with an external environmentsensor, such as a camera, and recognizes the external environmentinformation with an external environment recognition device. Moreparticularly, it relates to an external environment recognition systemthat corrects the direction in which the external environmentrecognition device performs recognition.

BACKGROUND ART

Japanese Laid-Open Patent Publication No. 2011-169728 discloses a devicethat corrects a yaw rate detected by a yaw rate sensor. The deviceestimates the yaw rate of the host vehicle based on image informationcaptured by a vehicle-mounted camera, and uses the estimated yaw rate tocorrect the yaw rate detected by the yaw rate sensor.

Image information captured by the camera is recognized by an externalenvironment recognition device (for example, a recognition ECU).Typically, the optical axis direction of the camera is aligned with thedirection to be recognized (a direction in the external environmentwhich is to be recognized), after which the optical axis direction ofthe camera is set as a reference direction of the external environmentrecognition device (the direction of recognition).

SUMMARY OF INVENTION

The posture of the camera changes due to vibration of the vehicle,contact with an occupant, and the like. Along with a change in theposture of the camera, the optical axis direction of the camera and thereference direction of the external environment recognition devicebecome misaligned with the direction to be recognized. This causes anerror in position information of a recognition target determined from arecognition result of the external environment recognition device.Japanese Laid-Open Patent Publication No. 2011-169728 does not take intoconsideration correcting of a recognition result of the externalenvironment recognition device.

The present invention was made in view of such a challenge and an objectthereof is to provide an external environment recognition system capableof accurately correcting a reference direction established for anexternal environment recognition device and accurately recognizing thepositions of recognition targets around a vehicle.

An external environment recognition system according to a first aspectof the present invention includes: a satellite positioning deviceconfigured to measure a position of a vehicle by receiving radio wavestransmitted from artificial satellites; a vehicle sensor configured todetect or estimate a behavior of the vehicle; an external environmentsensor configured to acquire external environment information around thevehicle; and an external environment recognition device configured torecognize the external environment information centered in a referencedirection, wherein the external environment recognition device isconfigured to correct a detected value of the vehicle sensor based onpositions of the vehicle in temporal sequence measured by the satellitepositioning device and correct the reference direction based on thecorrected detected value.

In the first aspect, the external environment recognition device may beconfigured to determine a traveled trajectory of the vehicle based onthe positions of the vehicle in temporal sequence measured by thesatellite positioning device, correct the detected value of the vehiclesensor based on the traveled trajectory, and correct the referencedirection based on the corrected detected value.

With the configuration above, the detected value of the vehicle sensoris initially corrected using the satellite positioning device, whichprovides position measurement of high accuracy, followed by correctionof the reference direction of the external environment recognitiondevice using the corrected detected value of the vehicle sensor. Thus, amisalignment in the reference direction established for the externalenvironment recognition device can be accurately corrected. As a result,it becomes possible to accurately recognize the positions of recognitiontargets around the vehicle.

An external environment recognition system according to a second aspectof the present invention includes: a satellite positioning deviceconfigured to measure a position of a vehicle by receiving radio wavestransmitted from artificial satellites; a storage device configured tostore a position of a ground object; an external environment sensorconfigured to acquire external environment information for the vehicle;and an external environment recognition device configured to recognizethe external environment information centered in a reference direction,wherein the external environment recognition device is configured tocorrect the reference direction based on the position of the vehiclemeasured by the satellite positioning device and on the position of theground object stored in the storage device.

In the second aspect, the external environment recognition device may beconfigured to determine a relative direction of the ground object withrespect to the vehicle based on the position of the vehicle measured bythe satellite positioning device and on the position of the groundobject stored in the storage device, and correct the reference directionbased on the relative direction.

With the configuration above, a misalignment in the reference directionestablished for the external environment recognition device can beaccurately corrected because the reference direction of the externalenvironment recognition device is corrected using the satellitepositioning device, which provides position measurement of highaccuracy. As a result, it becomes possible to accurately recognize thepositions of recognition targets around the vehicle.

In the first or second aspect, the external environment recognitiondevice may be configured to perform processing to correct the referencedirection when reliability of measurement by the satellite positioningdevice is higher than a predetermined reliability. With thisconfiguration, a misalignment in the reference direction established forthe external environment recognition device can be corrected moreaccurately.

The present invention can accurately correct a misalignment in thereference direction established for the external environment recognitiondevice. As a result, it becomes possible to accurately recognize thepositions of recognition targets around the vehicle.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a configuration of a vehicle controlsystem that includes an external environment recognition systemaccording to an embodiment of the present invention;

FIG. 2 is a flowchart of correction processing according to a firstembodiment;

FIG. 3 is a diagram comparing a traveled trajectory calculated based onthe position of travel with a yaw rate calculated based on the yaw rate;

FIG. 4A illustrates various directions, the imaging range of a camera,and the recognition zone of an external environment recognition devicewhen the posture of the camera is proper, FIG. 4B illustrates variousdirections, the imaging range of the camera, and the recognition zone ofthe external environment recognition device when the posture of thecamera is not proper, and FIG. 4C illustrates a reference direction andthe recognition zone after correction of the external environmentrecognition device;

FIG. 5A illustrates an image and the recognition zone captured with theproper posture of the camera, and FIG. 5B illustrates an image and therecognition zone captured with an improper posture of the camera;

FIG. 6 is a flowchart of correction processing according to a variant ofthe first embodiment;

FIG. 7 is a flowchart of correction processing according to a secondembodiment;

FIG. 8A illustrates the optical axis of a camera with a proper pan angleand a ground object, and FIG. 8B illustrates the optical axis of acamera with an improper pan angle and a ground object;

FIG. 9A illustrates the optical axis of a camera with a proper pitchangle and a ground object, and FIG. 9B illustrates the optical axis of acamera with an improper pitch angle and a ground object; and

FIG. 10A illustrates a ground object captured with a proper roll angle,and FIG. 10B illustrates a ground object captured with an improper rollangle.

DESCRIPTION OF EMBODIMENTS

An external environment recognition system according to the presentinvention is described below by showing preferred embodiments withreference to the accompanying drawings.

1 Configuration of Vehicle Control System 10

The external environment recognition system according to the presentinvention forms part of a vehicle control system 10 mounted on a vehicle100. In the following, the vehicle control system 10 is described withdescription of an external environment recognition system 12.

1.1 Overall Configuration

With reference to FIG. 1, the vehicle control system 10 is described.The vehicle control system 10 is incorporated in the vehicle 100 andcontrols the travel of the vehicle 100 by automated driving or manualdriving. The term “automated driving” is a concept that encompasses notonly “fully automated driving”, which controls the travel of the vehicle100 entirely automatically, but also “partially automated driving” and“driving assistance”, which control travel partially automatically.

The vehicle control system 10 basically includes an input system devicegroup, an external environment recognition device 22, a vehicle controldevice 28, and an output system device group. The devices that form theinput system and output system device groups are connected with theexternal environment recognition device 22 and/or with the vehiclecontrol device 28 via a communication line. The external environmentrecognition device 22 and the vehicle control device 28 are connectedwith each other via a communication line.

The input system device group includes external environment sensors 14,a navigation device 16, a communication device 18, a vehicle sensor 20,an automated driving switch 24, and an operation detection sensor 26connected with operation devices (not shown). The output system devicegroup includes a driving force device 30 for driving wheels (not shown),a steering device 32 for steering the wheels, a braking device 34 forbraking the wheels, and a notification device 36 that notifies a drivermainly visually, audibly, or tactilely.

Some of the input system devices (the external environment sensors 14,the navigation device 16, the communication device 18, and the vehiclesensor 20) and the external environment recognition device 22 form theexternal environment recognition system 12.

1.2 Specific Configuration of Input System Device Group

The external environment sensors 14 acquire information indicating anexternal environment state of the vehicle 100 (hereinafter, externalenvironment information) and outputs the external environmentinformation to the external environment recognition device 22.Specifically, the external environment sensors 14 include one or morecameras 40, one or more radars 42, and one or more LIDARs 44 (LightDetection and Ranging, Laser Imaging Detection and Ranging). At thestage of shipment of the vehicle 100, the detection direction of eachsensor [such as an optical axis direction 92 of the camera 40 (see FIG.4A)] is defined, for example, as a relative direction to a front-backdirection 90 of the vehicle 100. In this embodiment, the detectiondirection of each sensor coincides with (is parallel to) the front-backdirection 90 of the vehicle 100.

The navigation device 16 includes a satellite positioning device 46, anavigation storage device 48, and user interfaces, not illustrated (forexample, a touch panel display, a speaker, and a microphone). Thenavigation device 16 uses information detected by the satellitepositioning device 46, the vehicle sensor 20, and the like to measurethe current position (the position of travel) of the vehicle 100, andgenerates a travel route from that position to a destination specifiedby the user.

The satellite positioning device 46 measures the current position of thevehicle 100 by receiving radio waves transmitted from artificialsatellites. The satellite positioning device 46 is able to measure thelatitude and longitude of the current position if it can communicatewith three artificial satellites simultaneously, and is able to measurethe altitude of the current position in addition to the latitude andlongitude if it can communicate with four or more artificial satellitessimultaneously. As there are more artificial satellites with whichcommunication is possible, the accuracy of position measurementincreases and its reliability is improved.

The navigation storage device 48 stores the travel route generated bythe navigation device 16 as route information 84 and also stores mapinformation 82. The map information 82 is acquired via the communicationdevice 18 or via a recording medium (not shown).

The communication device 18 is configured to be capable of communicationwith roadside equipment, other vehicles, and external devices includinga server, and transmits and receives information relating to trafficequipment (such as traffic signals), information relating to othervehicles, probe information, or the latest map information 82, forexample. The pieces of information are output to the externalenvironment recognition device 22 or to the vehicle control device 28.

The vehicle sensor 20 includes a yaw rate sensor 52 for detecting anangular speed about a vertical axis, as well as various sensors, notillustrated, such as a speed sensor for detecting a vehicle velocity(vehicle speed) V, an acceleration sensor for detecting an acceleration,a lateral acceleration sensor for detecting a lateral acceleration, anda direction sensor for detecting orientation and direction, and agradient sensor detecting a gradient. Signals detected by each sensorare output to the external environment recognition device 22 and/or thevehicle control device 28 and stored in their respective memory 64 andmemory 80 as host vehicle information 86.

The automated driving switch 24 is a button switch provided on asteering wheel, an instrument panel, and the like, for example. Theautomated driving switch 24 is configured to be capable of switchingbetween multiple driving modes through manual operation by a user,including the driver. The automated driving switch 24 outputs a modeswitching signal to the vehicle control device 28.

The operation detection sensor 26 detects the presence or absence of thedriver's operation, the amount of operation, and the position ofoperation on various operation devices, not illustrated, such as anaccelerator pedal, a steering wheel, a brake pedal, a shift lever, and adirection indicator lever, for example. The operation detection sensor26 outputs, as detection results, the amount of pressing on theaccelerator (accelerator opening amount), the amount of steeringoperation (the amount of steering), the amount of pressing on the brake,the shift position, the direction of a right/left turn, and the like tothe vehicle control device 28.

1.3 Specific Configuration of Output System Device Group

The driving force device 30 includes a driving force electronic controlunit (ECU) and drive sources including an engine and a traction motor.The driving force device 30 generates driving force (torque) for thetraveling of the vehicle 100 in accordance with a vehicle control valueoutput from a vehicle control unit 76 and transfers it to the wheels viaa transmission, or directly.

The steering device 32 includes an electric power steering system (EPS)ECU and an EPS actuator. The steering device 32 changes the orientationof the wheels (steered wheels) in accordance with a vehicle controlvalue output from the vehicle control unit 76.

The braking device 34 is an electric servo brake used in combinationwith a hydraulic brake, for example, and includes a brake ECU and abrake actuator. The braking device 34 brakes the wheels in accordancewith a vehicle control value output from the vehicle control unit 76.

The notification device 36 includes a notification ECU, a displaydevice, an audio device, and a haptic device. The notification device 36performs notification operation relating to automated driving or manualdriving in accordance with a notification command output from thevehicle control device 28. At the time of notification operation, thenotification ECU controls one or more of the display device, the audiodevice, and the tactile device. At that time, the notification ECU maychange the device to be operated and/or the operation itself inaccordance with the content of the notification.

1.4 Configuration of External Environment Recognition Device 22

The external environment recognition device 22 includes one or more ECUsand includes the memory 64 and various function implementing components.In this embodiment, the function implementing components are softwarefunctional components, which implement their functions by the executionof programs stored in the memory 64 by a central processing unit (CPU).The function implementing components may also be realized as hardwarefunctional components, formed of an integrated circuit such asfield-programmable gate array (FPGA). The function implementingcomponents include an external environment recognition unit 60 and acorrection processing unit 62.

The external environment recognition unit 60 recognizes static externalenvironment information around the vehicle 100 using the externalenvironment information acquired by the external environment sensors 14,the map information 82 from the navigation device 16, and the like, andgenerates external environment recognition information. The staticexternal environment information includes, for example, recognitiontargets such as lane markings, stop lines, traffic lights, trafficsigns, ground objects (real estate), travel possible zones, and passingzones or emergency bays. The external environment recognition unit 60recognizes dynamic external environment information around the vehicle100 using the external environment information acquired by the externalenvironment sensors 14, information received by the communication device18, and the like, and generates external environment recognitioninformation. The dynamic external environment information includes, forexample, obstacles such as parked or stopped vehicles, trafficparticipants such as pedestrians and other vehicles (includingbicycles), and traffic signals (the signal colors of traffic lights).The dynamic external environment information also includes informationon the moving direction of each recognition target. The externalenvironment recognition unit 60 recognizes the position of eachrecognition target based on a positioning result of the satellitepositioning device 46 and the map information 82 from the navigationdevice 16. Before shipment of the vehicle 100, a reference direction 94to be recognized by the external environment recognition device 22 isset to the same direction as the direction to be detected by each sensor(for example, the optical axis direction 92) (see FIG. 4A).

The correction processing unit 62 performs the process described belowin [2] and [3] to correct the reference direction 94. The memory 64stores various programs as well as reference information 66 andcorrection processing information 68. The reference information 66 isinformation indicating the reference direction 94, and is stored, forexample, as information on an amount of misalignment relative to thedirection of the optical axis direction 92. The correction processinginformation 68 includes external environment information which isacquired by the external environment sensors 14 when correctionprocessing on the reference direction 94 is performed, information onthe position of the vehicle 100 as measured by the satellite positioningdevice 46 and the time of measurement, and information detected by thevehicle sensor 20. Further, the memory 64 stores the position of avanishing point 102 which is recognized by the external environmentrecognition device 22 while the vehicle 100 is moving straight on astraight road, that is, the position of the vanishing point 102 within arecognition zone 98 (see FIG. 5A). This is called an initial position Piof the vanishing point 102.

[1.5 Configuration of Vehicle Control Device 28]

The vehicle control device 28 includes one or more ECUs and includes thememory 80 and various function implementing components, as with theexternal environment recognition device 22. The function implementingcomponents include an action planning unit 70, a trajectory generatingunit 72, a driving mode control unit 74, and a vehicle control unit 76.

The action planning unit 70 creates an action plan (a temporal sequenceof events) for each travel section based on the recognition result ofthe external environment recognition device 22 and updates the actionplan where necessary. Event types may include decelerating,accelerating, branching, merging, lane keeping, lane changing, andovertaking, for example. Here, “decelerating” and “accelerating” are theevents of decelerating and accelerating the vehicle 100, respectively.“Branching” and “merging” are the events of making the vehicle 100smoothly travel at a branching point and a merge point, respectively.“Lane changing” is the event of making the vehicle 100 change the travellane. “Overtaking” is the event of making the vehicle 100 overtakeanother vehicle ahead. “Lane keeping” is the event of making the vehicle100 travel so as not to deviate from the travel lane and is segmentedaccording to combination with a travel pattern. Specific examples of thetravel pattern include a constant speed travel, a following travel, adecelerated travel, a curve travel, and an obstacle avoiding travel.

The trajectory generating unit 72 uses the map information 82, the routeinformation 84, and the host vehicle information 86 retrieved from thememory 80 to generate a planned travel trajectory conforming to theaction plan created by the action planning unit 70. This planned traveltrajectory is data that indicates target behaviors in temporal sequence,more specifically, a data set in temporal sequence with the data unitseach including a position, a posture angle, speed,acceleration/deceleration, a curvature, a yaw rate, and a steeringangle.

The driving mode control unit 74 performs transition processing from themanual driving mode to the automated driving mode or from the automateddriving mode to the manual driving mode in accordance with a signaloutput from the automated driving switch 24. The driving mode controlunit 74 also performs transition processing from the automated drivingmode to the manual driving mode in accordance with a signal output fromthe operation detection sensor 26.

The vehicle control unit 76 determines vehicle control values forcontrolling the travel of the vehicle 100 in accordance with the plannedtravel trajectory generated by the trajectory generating unit 72. Then,the vehicle control unit 76 outputs the respective vehicle controlvalues it determined to the driving force device 30, the steering device32, and the braking device 34.

2 First Embodiment 2.1 Processing Performed by External EnvironmentRecognition System 12

With reference to FIG. 2, processing performed by the externalenvironment recognition system 12 of a first embodiment is described.The processing shown in FIG. 2 may be executed when the driver performsa given operation, executed at a given timing, such as at the time ofpowering up of the vehicle 100, or executed periodically. In the firstembodiment, a traveled trajectory of the vehicle 100 is used forcorrection of the reference direction 94. For calculating the traveledtrajectory of the vehicle 100, it is necessary for the vehicle 100 totravel a certain distance or more.

At step S1, the satellite positioning device 46 determines whether theaccuracy of position measurement is high or not. Specifically, itdetermines whether the number of artificial satellites from which radiowaves can be received is equal to or higher than a predetermined numberstored in the navigation storage device 48. Herein, it is preferablethat radio waves from at least four artificial satellites are received.If the number is equal to or higher than the predetermined number, thatis, if the reliability is high (step S1: YES), the process moves on tostep S2. By contrast, if the number is less than the predeterminednumber, that is, if the reliability is low (step S1: NO), the process atstep S1 is performed again.

At step S2, various kinds of information required for the correctionprocesses at steps S3 to S6 are acquired. The satellite positioningdevice 46 measures the position of the vehicle 100 while the vehicle 100travels a certain distance or more. The measurement result (position S)is stored in the memory 64 of the external environment recognitiondevice 22 as the correction processing information 68 in temporalsequence. Also, the yaw rate sensor 52 detects a yaw rate Y2 (see FIG.3). The detected value (yaw rate Y2) is stored in the memory 64 of theexternal environment recognition device 22 as correction processinginformation 68 in temporal sequence.

At step S3, the correction processing unit 62 calculates a traveledtrajectory L1 of the vehicle 100 (see FIG. 3) based on the measurementresult (position S) of the satellite positioning device 46 stored as thecorrection processing information 68. The traveled trajectory L1 shownin FIG. 3 is determined by linear interpolation using three measurementresults (positions S1 to S3). At step S4, the correction processing unit62 calculates a yaw rate Y1 based on the traveled trajectory L1. Forexample, it calculates the vehicle speed V of the vehicle 100 (theamount of positional change per unit time) based on the measurementresult of the satellite positioning device 46 and calculates the yawrate Y1 from a radius of curvature R of the traveled trajectory L1 andthe vehicle speed V (Y1=V/R).

At step S5, the correction processing unit 62 compares the yaw rate Y1determined by calculation with the yaw rate Y2 detected by the yaw ratesensor 52. If the yaw rate Y1 and the yaw rate Y2 are different, theprocess moves on to step S6. By contrast, if the yaw rate Y1 and the yawrate Y2 agree (substantially agree) with each other, the process moveson to step S7.

At step S6, the correction processing unit 62 corrects the detectedvalue of the yaw rate sensor 52. Here, regarding the yaw rate Y1 as thetrue value, the amount of disagreement of the yaw rate Y2 relative tothe yaw rate Y1 is used as a correction value Sa. The correction valueSa is a numerical value for converting the detected value of the yawrate sensor 52 to the true value. The correction processing unit 62stores the correction value Sa in the memory 64 and also outputs it tothe vehicle control device 28. The vehicle control device 28 stores thecorrection value Pa in the memory 80.

At step S7, various kinds of information are acquired again. The camera40 acquires image information, and the yaw rate sensor 52 detects a yawrate Y2A (=the value of the yaw rate Y2 after correction with thecorrection value Sa). At step S8, the correction processing unit 62detects the amount by which the vanishing point 102 recognized by theexternal environment recognition unit 60 when the vehicle 100 is movingstraight (when the yaw rate Y2A≈0) is shifted from the initial positionPi (the amount of displacement).

At step S9, the correction processing unit 62 compares the amount ofdisplacement of the vanishing point 102 with a misalignment tolerancestored in the memory 64. If the amount of displacement is equal to orgreater than the tolerance (step S9: YES), the process moves on to stepS10. By contrast, if the amount of displacement is less than thetolerance (step S9: NO), it is determined that there is no misalignmentin the reference direction 94, and the process ends.

At step S10, the correction processing unit 62 corrects the referencedirection 94 indicated by the reference information 66. Here, thereference direction 94 is corrected so that the position P of thevanishing point 102 in the recognition zone 98 coincides with theinitial position Pi. A way of correction for the case of misalignment inpan angle is illustrated below.

As shown in FIG. 4A, when the mounting posture (the pan angle) of thecamera 40 on the vehicle 100 is proper (when the camera 40 is in theinitial posture), the optical axis direction 92 of the camera 40 and thereference direction 94 of the external environment recognition unit 60coincide with the front-back direction 90 of the vehicle 100. Under thiscondition, the camera 40 captures image information of an imaging zone96 (FIG. 5A) centered in the optical axis direction 92, and the externalenvironment recognition unit 60 recognizes image information of therecognition zone 98 (FIG. 5A) centered in the reference direction 94.

As shown in FIG. 4B, when the mounting posture of the camera 40 on thevehicle 100 is not proper (when the camera 40 has been shifted from theinitial posture), an optical axis direction 92′ of the camera 40 and areference direction 94′ of the external environment recognition unit 60do not coincide with the front-back direction 90 of the vehicle 100.Under this condition, the camera 40 captures image information of animaging zone 96′ (FIG. 5B) centered in the optical axis direction 92′.The external environment recognition unit 60 recognizes imageinformation of a recognition zone 98′ (FIG. 5B) centered in thereference direction 94′. In the recognition zone 98′, the vanishingpoint 102 is not at the initial position Pi but at position P, displacedfrom the initial position Pi to the right.

The correction processing unit 62 determines a direction and an amountof correction for adjusting the initial position Pi in the recognitionzone 98′ to the position P of the current vanishing point 102. Then,based on the direction and amount of correction, it corrects thereference direction 94′ of the external environment recognition unit 60.As a result of the correction, the position P of the vanishing point 102within the recognition zone 98 coincides with the initial position Pi,as indicated by the broken lines in FIG. 5B. Also, as shown in FIG. 4C,the external environment recognition unit 60 is able to recognize imageinformation of the recognition zone 98 (FIG. 5B) centered in thereference direction 94, that is, the front-back direction 90 of thevehicle 100.

2.2 Summarization of the First Embodiment

The external environment recognition system 12 according to the firstembodiment includes: the satellite positioning device 46 configured tomeasure a position of the vehicle 100 by receiving radio wavestransmitted from artificial satellites; the vehicle sensor 20 (yaw ratesensor 52) configured to detect or estimate a behavior of the vehicle100; the external environment sensors 14 (camera 40) configured toacquire external environment information around the vehicle 100; and theexternal environment recognition device 22 configured to recognize theexternal environment information centered in the reference direction 94.The external environment recognition device 22 is configured to correctthe detected value Y2 of the vehicle sensor 20 (the yaw rate sensor 52)based on positions of the vehicle 100 in temporal sequence measured bythe satellite positioning device 46 and correct the reference direction94 based on the corrected detected value Y2A.

Specifically, the external environment recognition device 22 determinesa traveled trajectory L1 of the vehicle 100 based on the positions S1,S2, S3 of the vehicle 100 in temporal sequence measured by the satellitepositioning device 46 (step S3), corrects the detected value Y2 of thevehicle sensor 20 (the yaw rate sensor 52) based on the traveledtrajectory L1 (step S6), and corrects the reference direction 94 basedon the corrected detected value Y2A (step S10).

With the configuration above, the detected value Y2 of the vehiclesensor 20 (the yaw rate sensor 52) is initially corrected using thesatellite positioning device 46, which provides position measurement ofhigh accuracy, followed by correction of the reference direction 94 ofthe external environment recognition device 22 using the correcteddetected value Y2A of the vehicle sensor 20 (the yaw rate sensor 52).Thus, a misalignment in the reference direction 94 established for theexternal environment recognition device 22 can be accurately corrected.As a result, it becomes possible to accurately recognize the positionsof recognition targets around the vehicle 100.

The external environment recognition device 22 also performs process tocorrect the reference direction 94 when the reliability of measurementby the satellite positioning device 46 is higher than a predeterminedreliability. With this configuration, a misalignment in the referencedirection 94 established for the external environment recognition device22 can be corrected more accurately.

2.3 Variant 1 of the First Embodiment

With reference to FIG. 6, a variant of the first embodiment isdescribed. The external environment recognition system 12 may performthe operations shown in FIG. 6 instead of the operations shown in FIG.2. The series of processes shown in FIG. 6 and those shown in FIG. 2correspond with each other except for some processes. Herein, ones ofthe series of processes shown in FIG. 6 that are different from theseries of processes shown in FIG. 2 are described.

At step S24 shown in FIG. 6, the correction processing unit 62calculates a traveled trajectory L2 (see FIG. 3) based on the yaw rateY2 detected by the yaw rate sensor 52. Then, at step S25, the correctionprocessing unit 62 compares the traveled trajectory L1 calculated basedon the positioning result of the satellite positioning device 46 withthe traveled trajectory L2 calculated based on the yaw rate Y2 detectedby the yaw rate sensor 52. This process may be performed in place of thesteps S4 and S5 of FIG. 2.

2.4 Variant 2 of the First Embodiment

It is also possible in the first embodiment to correct some othervehicle sensor 20, for example, the speed sensor, the accelerationsensor, the lateral acceleration sensor, and the like, using thecorrected yaw rate Y2A. The reason for initially correcting the detectedvalue of the yaw rate sensor 52 in the first embodiment is that the yawrate can be corrected most accurately. However, some other vehiclesensor 20 may be corrected first. Also, in addition to the optical axisdirection 92 of the camera 40, the optical axis directions of theradar(s) 42 and of the LIDAR(s) 44 may be corrected.

3 Second Embodiment 3.1 Operation of External Environment RecognitionSystem 12

With reference to FIG. 7, process performed by the external environmentrecognition system 12 according to a second embodiment is described. Asin the first embodiment, the process shown in FIG. 7 may be executedwhen the driver performs a given operation, executed at a given timing,such as at the time of powering up of the vehicle 100, or executedperiodically.

At step S41, the satellite positioning device 46 determines whether theaccuracy of position measurement is high or not. Specifically, itdetermines whether the number of artificial satellites from which radiowaves can be received is equal to or higher than a predetermined numberstored in the navigation storage device 48. If the number is equal to orhigher than the predetermined number, that is, if the reliability ishigh (step S41: YES), the process moves on to step S42. By contrast, ifthe number is less than the predetermined number, that is, if thereliability is low (step S41: NO), the process at step S41 is performedagain.

At step S42, various kinds of information required for the correctionprocess at steps S43 to S46 are acquired. The camera 40 acquires imageinformation, and the satellite positioning device 46 measures theposition of the vehicle 100.

At step S43, the correction processing unit 62 calculates the relativedirection of a ground object 110 (see FIG. 5A, for instance) (thedirection of the ground object 110 with respect to the position S of thevehicle 100) based on the positioning result (position S) obtained bythe satellite positioning device 46 and on the map information 82 storedin the navigation storage device 48. The direction calculated here iscalled first direction D1. This process is performed when the externalenvironment recognition unit 60 is able to recognize the ground object110 within a predetermined range around the vehicle 100 based on the mapinformation 82.

At step S44, the external environment recognition unit 60 recognizes theground object 110 based on the image information, and the correctionprocessing unit 62 calculates the relative direction of the groundobject 110 recognized by the external environment recognition unit 60(the direction of the ground object 110 with respect to the position Sof the vehicle 100). The direction calculated here is called seconddirection D2.

At step S45, the first direction D1 and the second direction D2 arecompared. If the first direction D1 and the second direction D2 aredifferent (step S45: YES), the process moves on to step S46. Bycontrast, if the first direction D1 and the second direction D2 coincide(substantially coincide) with each other (step S45: NO), it isdetermined that there is no misalignment in the reference direction 94,and the process ends.

At step S46, the correction processing unit 62 corrects the referencedirection 94 indicated by the reference information 66. Here, regardingthe first direction D1 as the true value, a direction of correction andan amount of correction for making the second direction D2 coincide withthe first direction D1, that is, the true value, are calculated. Usingthe direction of correction and the amount of correction, the referencedirection 94 is corrected.

As shown in FIGS. 8A, 9A, and 10A, when the mounting posture (the panangle, pitch angle, and roll angle) of the camera 40 on the vehicle 100is proper, the optical axis direction 92 of the camera 40 and thereference direction 94 of the external environment recognition unit 60coincide with the front-back direction 90 of the vehicle 100. Under thiscondition, the camera 40 captures image information of the imaging zone96 (FIG. 5A) centered in the optical axis direction 92, and the externalenvironment recognition unit 60 recognizes image information of therecognition zone 98 (FIG. 5A) centered in the reference direction 94.

As shown in FIGS. 8B, 9B, and 10B, when the mounting posture of thecamera 40 on the vehicle 100 is not proper, the optical axis direction92′ of the camera 40 and the reference direction 94′ of the externalenvironment recognition unit 60 do not coincide with the front-backdirection 90 of the vehicle 100. Under this condition, the camera 40captures image information of the imaging zone 96′ (FIG. 5B) centered inthe optical axis direction 92′, and the external environment recognitionunit 60 recognizes image information of the recognition zone 98′ (FIG.5B) centered in the reference direction 94′ (FIG. 5B shows a recognitionzone 98′ with misalignment only in the pan angle).

The correction processing unit 62 corrects the reference direction 94′of the external environment recognition unit 60 based on the position ofthe ground object 110 so as to coincide with the front-back direction 90of the vehicle 100. As a result of the correction, the externalenvironment recognition unit 60 is able to recognize image informationof the recognition zone 98 (FIG. 5B) centered in the reference direction94, that is, the front-back direction 90 of the vehicle 100.

3.2 Summarization of the Second Embodiment

The external environment recognition system 12 according to the secondembodiment includes: the satellite positioning device 46 configured tomeasure a position of the vehicle 100 by receiving radio wavestransmitted from artificial satellites; the navigation storage device 48configured to store a position of the ground object 110; the externalenvironment sensors 14 (camera 40) configured to acquire externalenvironment information for the vehicle 100; and the externalenvironment recognition device 22 configured to recognize the externalenvironment information centered in the reference direction 94. Theexternal environment recognition device 22 is configured to correct thereference direction 94 based on the position of the vehicle 100 measuredby the satellite positioning device 46 and on the position of the groundobject 110 stored in the navigation storage device 48.

Specifically, the external environment recognition device 22 maydetermine the relative direction of the ground object 110 with respectto the vehicle 100 based on the position of the vehicle 100 measured bythe satellite positioning device 46 and on the position of the groundobject 110 stored in the navigation storage device 48, and correct thereference direction 94 based on the relative direction.

With the configuration above, a misalignment in the reference direction94 established for the external environment recognition device 22 can beaccurately corrected because the reference direction 94 of the externalenvironment recognition device 22 is corrected using the satellitepositioning device 46, which provides position measurement of highaccuracy. As a result, it becomes possible to accurately recognize thepositions of recognition targets around the vehicle 100.

As with the first embodiment, the external environment recognitiondevice 22 performs process to correct the reference direction 94 whenthe reliability of measurement by the satellite positioning device 46 ishigher than a predetermined reliability. With this configuration, amisalignment in the reference direction 94 established for the externalenvironment recognition device 22 can be corrected more accurately.

In the process at step S44 of FIG. 7, the relative direction of theground object 110 is measured most easily when the vehicle 100 is in aparked state. Thus, the processes of the second embodiment arepreferably performed when the vehicle 100 is in the parked state.

It will be apparent that the external environment recognition systemaccording to the present invention is not limited to the aboveembodiments but may adopt various other configurations without departingfrom the scope of the present invention.

The operation of the external environment recognition system 12 may bestopped if the amount of misalignment in the reference direction 94 hasreached a certain amount or more, for example, if a ground object 110which fits within a predetermined area (such as a center area) of theimaging zone 96 when there is no misalignment in the reference direction94 has ceased to fit in the imaging zone 96, or if such a ground object110 is positioned at an edge of the imaging zone 96.

1. An external environment recognition system comprising: a satellite positioning device configured to measure a position of a vehicle by receiving radio waves transmitted from artificial satellites; a vehicle sensor configured to detect or estimate a behavior of the vehicle; an external environment sensor configured to acquire external environment information around the vehicle; and an external environment recognition device configured to recognize the external environment information centered in a reference direction, wherein the external environment recognition device is configured to correct a detected value of the vehicle sensor based on positions of the vehicle in temporal sequence measured by the satellite positioning device and correct the reference direction based on the corrected detected value.
 2. The external environment recognition system according to claim 1, wherein the external environment recognition device is configured to determine a traveled trajectory of the vehicle based on the positions of the vehicle in temporal sequence measured by the satellite positioning device, correct the detected value of the vehicle sensor based on the traveled trajectory, and correct the reference direction based on the corrected detected value.
 3. An external environment recognition system comprising: a satellite positioning device configured to measure a position of a vehicle by receiving radio waves transmitted from artificial satellites; a storage device configured to store a position of a ground object; an external environment sensor configured to acquire external environment information for the vehicle; and an external environment recognition device configured to recognize the external environment information centered in a reference direction, wherein the external environment recognition device is configured to correct the reference direction based on the position of the vehicle measured by the satellite positioning device and on the position of the ground object stored in the storage device.
 4. The external environment recognition system according to claim 3, wherein the external environment recognition device is configured to determine a relative direction of the ground object with respect to the vehicle based on the position of the vehicle measured by the satellite positioning device and on the position of the ground object stored in the storage device, and correct the reference direction based on the relative direction.
 5. The external environment recognition system according to claim 1, wherein the external environment recognition device is configured to perform processing to correct the reference direction when reliability of measurement by the satellite positioning device is higher than a predetermined reliability.
 6. The external environment recognition system according to claim 3, wherein the external environment recognition device is configured to perform processing to correct the reference direction when reliability of measurement by the satellite positioning device is higher than a predetermined reliability. 